Imaging device, imaging method, and imaging program

ABSTRACT

Provided are an imaging device, an imaging method, and an imaging program capable of easily acquiring a slow moving image with good image quality. In one aspect of the present invention, an imaging device includes an optical system, an imaging element, and a processor, and the processor performs detection processing of detecting a movement of a subject based on an image signal output from the imaging element, frame rate control of increasing a frame rate of a moving image output from the imaging element based on the detected movement, exposure control processing of maintaining a rate of an exposure time per frame of the moving image constant according to the increase in the frame rate, and dimming control processing of changing a degree of dimming of the optical system according to the exposure control processing.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of and claims thepriority benefit of U.S. patent application Ser. No. 17/899,621, filedon Aug. 31, 2022, now allowed, which is a continuation of PCTInternational Application No. PCT/JP2021/011307 filed on Mar. 19, 2021claiming priority under 35 U.S.C § 119(a) to Japanese Patent ApplicationNo. 2020-053920 filed on Mar. 25, 2020. Each of the above applicationsis hereby expressly incorporated by reference, in its entirety, into thepresent application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to an imaging device that captures amoving image, an imaging method, and an imaging program.

2. Description of the Related Art

Regarding a technique for capturing a moving image, for example, animaging device that controls a transmittance of a filter according to achange in a frame rate of imaging is described in WO2015/151813A.

SUMMARY OF THE INVENTION

One embodiment according to the technique of the present disclosureprovides an imaging device, an imaging method, and an imaging programcapable of easily acquiring a slow moving image with good image qualityfor a moving subject.

An imaging device according to a first aspect of the present inventioncomprises an optical system, an imaging element, and a processor. Theprocessor performs detection processing of detecting a movement of asubject based on an image signal output from the imaging element, framerate control of increasing a frame rate of a moving image output fromthe imaging element based on the detected movement, exposure controlprocessing of maintaining a rate of an exposure time per frame of themoving image constant according to the increase in the frame rate, anddimming control processing of changing a degree of dimming of theoptical system according to the exposure control processing.

In the imaging device according to a second aspect in the first aspect,the processor calculates a first timing for starting to change the framerate and the exposure time and a second timing for setting the framerate and the exposure time as target values based on the movement,increases the frame rate from the first timing to the second timing, andshortens the exposure time from the first timing to the second timing.

In the imaging device according to a third aspect in the second aspect,the processor calculates a timing at which a distance between aplurality of subjects is equal to or less than a threshold value as thesecond timing based on the detected movement in the frame rate control.

In the imaging device according to a fourth aspect in the third aspect,the processor sets a rate of change in the frame rate and the exposuretime according to a relative speed between the plurality of subjects.

In the imaging device according to a fifth aspect in any one of thesecond to fourth aspects, the processor sets the target values for theframe rate and the exposure time.

In the imaging device according to a sixth aspect in any one of thesecond to fifth aspects, the processor returns values of the frame rate,the exposure time, and the degree of dimming to values set beforestarting the frame rate control, the exposure control processing, andthe dimming control processing after the second timing has elapsed.

In the imaging device according to a seventh aspect in any one of thefirst to sixth aspects, the processor captures a first moving image,which is a moving image of the increased frame rate after the firsttiming, by setting a bit rate higher as the frame rate is higher.

In the imaging device according to an eighth aspect in the seventhaspect, the processor sets an exposure time per frame to half or more ofa frame interval for the first moving image.

In the imaging device according to a ninth aspect in the seventh oreighth aspect, the processor displays the first moving image on adisplay device at the same initial frame rate as before starting theframe rate control.

In the imaging device according to a tenth aspect in the ninth aspect,the processor thins out a frame of the first moving image to generate asecond moving image having the initial frame rate and displays the firstmoving image and the second moving image on the display device at theinitial frame rate.

In the imaging device according to an eleventh aspect in any one of thefirst to tenth aspects, the processor outputs information prompting auser to set at least one of a frame rate, an exposure time, a degree ofdimming of a dimming element, a stop, a sensitivity, or a bit rate.

In the imaging device according to a twelfth aspect in any one of thefirst to eleventh aspects, the optical system has a stop mechanism inwhich a degree of stop is variable. The processor changes the degree ofstop of the stop mechanism to a release side in accordance withshortening of the exposure time.

An imaging method according to a thirteenth aspect of the presentinvention is an imaging method by an imaging device including an opticalsystem, an imaging element, and a processor. Processing performed by theprocessor comprises a detection step of detecting a movement of asubject based on an image signal output from the imaging element, aframe rate control step of continuously increasing a frame rate of amoving image output from the imaging element based on the detectedmovement, an exposure control step of maintaining a rate of an exposuretime per frame of the moving image constant according to the increase inthe frame rate, and a dimming control step of changing a degree ofdimming of the optical system according to the control of the exposuretime. The imaging method according to the thirteenth aspect may furtherinclude the same configurations as those of the second to twelfthaspects.

An imaging program according to a fourteenth aspect of the presentinvention causes a computer to execute the imaging method according tothe thirteenth aspect. A non-transitory recording medium recording acomputer-readable code of the program can also be mentioned as an aspectof the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a schematic configuration of an imagingdevice according to a first embodiment.

FIG. 2 is a diagram showing a functional configuration of an imageprocessing device.

FIG. 3 is a flowchart showing processing of an imaging method accordingto the first embodiment.

FIG. 4 is a flowchart (continuation of FIG. 3 ) showing the processingof the imaging method according to the first embodiment.

FIGS. 5A and 5B are diagrams showing a state of calculating an eventtiming based on a movement of a subject.

FIGS. 6A to 6C are graphs showing a state of controlling a frame rateand the like.

FIG. 7 is a graph showing a state of controlling stop.

FIGS. 8A to 8C are other graphs showing a state of controlling the framerate and the like.

FIG. 9 is a graph showing a state of controlling the frame rate based ona relative speed of the subject.

FIGS. 10A and 10B are diagrams showing event occurrence by a movingsubject and a stationary subject.

FIG. 11 is a flowchart showing processing of Modification Example 6.

FIGS. 12A to 12C are diagrams showing a state in which a first movingimage and a second moving image are displayed at the same time.

FIGS. 13A and 13B are external views of a smartphone according to asecond embodiment.

FIG. 14 is a block diagram showing a schematic configuration of thesmartphone.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

One embodiment of an imaging device, an imaging method, and an imagingprogram according to the present invention is as follows. In thedescription, accompanying drawings will be referred to as necessary.

First Embodiment

<Overall Configuration of Imaging Device>

FIG. 1 is a diagram showing a schematic configuration of a camera 100(imaging device and optical system) according to a first embodiment. Thecamera 100 is configured of an interchangeable lens 200 (optical system)and an imaging device body 300 (imaging device body) and forms a subjectimage (optical image) on an imaging element 310 with an imaging lensincluding a zoom lens 220 and a focus lens 240 described below. Theinterchangeable lens 200 and the imaging device body 300 can be attachedand detached through a mount (not shown). Although the case where thecamera 100 comprises the interchangeable lens 200 is described in thefirst embodiment, a lens device (optical system) may be fixed to acamera body in the present invention.

<Configuration of Interchangeable Lens>

The interchangeable lens 200 comprises a dimming element 210, a zoomlens 220, a stop 230, a focus lens 240, and a lens drive unit 250. Thelens drive unit 250 drives the zoom lens 220 and the focus lens 240forward and backward in response to a command from an image processingdevice 340 (lens drive control unit 360 in FIG. 2 : processor) toperform zoom (optical zoom) adjustment and focus adjustment. The zoomadjustment and the focus adjustment may be performed in response to azoom operation and a focus operation (moving rotationally with a zoomring and a focus ring (not shown) or the like) performed by a user, inaddition to the command from the image processing device 340. Further,the lens drive unit 250 changes a degree of dimming of the dimmingelement 210 (dimming element, dimming filter, and optical system) inresponse to the command from the image processing device 340 (lens drivecontrol unit 360). Furthermore, the lens drive unit 250 controls thestop 230 (stop mechanism) in response to the command from the imageprocessing device 340 to adjust exposure. On the other hand, informationsuch as positions of the zoom lens 220 and the focus lens 240 and anopening degree of the stop 230 is input to the image processing device340. The interchangeable lens 200 has an optical axis L.

<Configuration of Dimming Element And Stop>

The dimming element 210 is configured of, for example, a variableneutral density (ND) filter and can change the degree of dimming. Thedegree of dimming can be changed, for example, by stacking andrelatively rotating a plurality of ND filters having different degreesof dimming in a rotation direction. Alternatively, a dimming element maybe configured by using an electrochromic element, and a voltage appliedto the element may be changed to change the degree of dimming. Further,the dimming element may be configured by using a liquid crystal displayelement, and a liquid crystal transmittance may be changed to change thedegree of dimming. The stop 230 (stop mechanism) is configured of avane-shaped member or a plate-shaped member that is movable and/orrotatable and can change a degree of stop. A stop having a variable stopmay be configured by using a liquid crystal display element. AlthoughFIG. 1 describes an aspect in which the dimming element 210 and the stop230 are provided in the interchangeable lens 200, the dimming elementand/or the stop may be provided on the imaging device body 300 side.

<Configuration of Imaging Device Body>

The imaging device body 300 comprises the imaging element 310 (imagingelement), an analog front end (AFE) 320, an analog to digital (A/D)converter 330, and the image processing device 340 (processor, conditionsetting unit, detection processing unit, frame rate control unit,exposure control processing unit, imaging unit, dimming controlprocessing unit, stop control unit, display control unit, informationoutput unit, and lens drive control unit). Further, the imaging devicebody 300 comprises an operation unit 370, a recording device 380(recording device), a monitor 390 (display device), and a speaker 395.The imaging device body 300 may have a shutter (not shown) for blockinglight transmitted through the imaging element 310. The shutter may be amechanical shutter or an electronic shutter. In a case of the electronicshutter, the image processing device 340 controls a charge accumulationperiod of the imaging element 310, and thus an exposure time (shutterspeed) can be adjusted.

The imaging element 310 comprises a light receiving surface in which alarge number of light receiving elements are arranged in a matrix. Then,subject light transmitted through the dimming element 210 (dimmingelement, dimming filter, and optical system), the zoom lens 220 (opticalsystem), the stop 230 (stop mechanism and optical system), and the focuslens 240 (optical system) is formed on the light receiving surface ofthe image element 310 and is converted into an electric signal by eachlight receiving element. A color filter of R (red), G (green), or B(blue) is provided on the light receiving surface of the imaging element310, and a color image of a subject can be acquired based on a signal ofeach color. Various photoelectric conversion elements such ascomplementary metal-oxide semiconductor (CMOS) and charge-coupled device(CCD) can be used as the imaging element 310. The AFE 320 performs noiseremoval, amplification, and the like of an analog image signal outputfrom the imaging element 310, and the A/D converter 330 converts thecaptured analog image signal into a digital image signal with agradation width.

In a case where the imaging element 310 is the CMOS, the AFE 320 and theA/D converter 330 are often included in the imaging element 310. As anexample of using the CMOS, the A/D converter 330 built in the imagingelement 310 converts the analog image signal output from the imagingelement 310 into the digital image signal. Thereafter, a digitalprocessing unit (not shown) built in the imaging element 310 performsdigital sampling two correlation pile, digital gain processing,correction processing, and the like to convert the analog image signalinto the digital image signal.

<Configuration of Image Processing Device>

FIG. 2 is a diagram showing a functional configuration of the imageprocessing device 340 (processor). The image processing device 340comprises a condition setting unit 342, a detection processing unit 344,a frame rate control unit 346, an exposure control processing unit 348,an imaging unit 350, a dimming control processing unit 352, a stopcontrol unit 354, a display control unit 356, an information output unit358, and the lens drive control unit 360. The image processing device340 performs processing such as moving image capturing (imaging) andgeneration and still image capturing (imaging) and generation based onthe digital image signal input from the A/D converter 330. Theprocessing by the image processing device 340 will be described below indetail.

Functions of the image processing device 340 can be realized by usingvarious processors. The various processors include, for example, acentral processing unit (CPU), which is a general-purpose processor thatexecutes software (program) to realize various functions. Further, thevarious processors described above include a graphics processing unit(GPU), which is a processor specialized in image processing. Further,the various processors described above include a programmable logicdevice (PLD), which is a processor whose circuit configuration can bechanged after manufacturing such as a field programmable gate array(FPGA). Furthermore, the various processors described above include adedicated electric circuit, which is a processor having a circuitconfiguration specially designed for executing specific processing suchas an application specific integrated circuit (ASIC).

Each function of the image processing device 340 may be realized by oneprocessor or may be realized by a plurality of processors. Further, oneprocessor may support a plurality of functions. Furthermore, eachfunction of the image processing device 340 may be realized by acircuit, or a part of each function may be realized by a circuit and therest may be realized by a processor.

In a case where the above processor or circuitry executes the software(program), a processor (computer) readable code of the executed softwareis stored in a non-transitory recording medium such as a read onlymemory (ROM). The processor then refers to the software. The softwarestored in the non-transitory recording medium includes an imagingprogram (program for operating imaging device) for executing an imagingmethod according to the present invention. The code may be recorded invarious magneto-optical storages and a non-transitory recording mediumsuch as a semiconductor memory, instead of the ROM. In a case where theprocessing is performed by using the software, for example, a randomaccess memory (RAM) may be used as a temporary storage area, and forexample, data stored in an electronically erasable and programmable readonly memory (EEPROM) (not shown) may also be referred to.

The image processing device 340 comprises a ROM 362 (non-transitoryrecording medium, memory) in addition to the above-mentioned parts. TheROM 362 records a computer (for example, various processors constitutingimage processing device 340) readable code of a program (includingprogram for executing imaging method according to the present invention)necessary for capturing, recording, displaying, and the like of theimage. Further, the image processing device 340 comprises a temporarystorage area and a RAM 364 (memory) as a work area.

<Operation Unit>

The operation unit 370 has a release button, a button for operation (forexample, a cross button, a Quick button, an OK button, or the like), adial, a switch, and the like, which are not shown, and the user canperform various operations such as an imaging mode setting, an imagingcondition setting, a moving image capturing instruction, and a stillimage capturing instruction. Further, the image processing device 340can receive these user instructions. The monitor 390 (display device)may be configured of a touch panel type device and used as an operationunit.

<Recording Device>

The recording device 380 (recording device) is configured of variousmagneto-optical recording media, a non-transitory recording medium suchas a semiconductor memory, and a control circuit thereof and storesmoving images (first moving image, second moving image, normal movingimage, and the like), still images, and the like. As a recording medium,a type capable of being attached to and detached from the imaging devicebody 300 can be used. The captured images (moving image and still image)may be transmitted to and stored in an external (other than recordingdevice 380) recording medium or recording device by, for example, wiredand/or wireless communication.

<Monitor and Speaker>

The monitor 390 (display device) is configured of, for example, a touchpanel type liquid crystal display panel and can display the movingimages (normal moving image and slow moving image), the still image, amessage to the user (for example, information prompting user to set atleast one of frame rate, exposure time, dimming degree of dimmingelement, stop, sensitivity, or bit rate), and the like. The monitor 390can be disposed on a back surface side, a top surface side, or the likeof the imaging device body 300. This message may be output by voice fromthe speaker 395. The camera 100 may be provided with an optical and/orelectronic viewfinder.

<Problem in Slow Imaging and Method of Present Invention>

In general, an exposure time of half or more of the frame rate isconsidered to be appropriate as the exposure time of the moving image(example: exposure time of 1/120 sec or more for a 60 frame per second(fps). In a case where the exposure time thereof is shorter than theexposure time of half or more of the frame rate, a phenomenon in which amovement of the subject becomes jerky and intermittent (so-called “senseof ruffling”) occurs, which is said to be unsuitable for the movingimage. In order to extend the exposure time while maintainingappropriate exposure, the sensitivity may be reduced or the stop may benarrowed down. However, there is a limit on the reduction of thesensitivity, and the moving image may be different from the imagingperson's intention due to a change in a depth of field in a case wherethe stop is narrowed down. For this reason, an expensive imaging devicesuch as a device for movie imaging is equipped with a function capableof reducing an amount of light with a built-in ND filter or the like toextend the exposure time while maintaining the sensitivity and stopconditions constant.

In addition, there is a known function called “slow imaging” in whichthe imaging is performed at a frame rate (high frame rate) higher than anormal frame rate at the time of imaging and reproduction is performedat the normal frame rate at the time of reproduction (example: imagingat 240 fps and reproduction at 60 fps make 4 times slow). Hereinafter,the moving image captured by this function is referred to as “slowmoving image”.

Since the frame rate is high in the slow imaging, the imaging needs tobe performed with a high-speed shutter (short exposure time) accordingto the general imaging method described above (exposure time of 1/480sec or more at 240 fps). In a case where a state where the imaging isperformed with the appropriate exposure at the normal frame rate isswitched to the slow imaging such that the sense of ruffling does notoccur, the exposure time needs to be significantly changed in accordancewith the switching to the high frame rate. However, since the dimmingdegree of the ND filter cannot be changed in an instant (short time),the sensitivity that can be changed in an instant (short time) isswitched. However, with the switching of the sensitivity, “sense ofgranularity” (presence or absence, degree, pattern, and the like ofgranular noise in image) may change. In the imaging in sports and thelike, the switching of the sensitivity is used with the knowledge thatthe sense of granularity may change since retaking is not possible.However, the change in the sense of granularity is more problematic inmovie imaging and the like in which retaking is possible. Thus, ingeneral, the amount of light is adjusted with the ND filter to make thesensitivity uniform, and a normal frame rate portion and a high framerate portion are separately imaged.

However, such multiple times of imaging take time and effort and cannotbe employed in a case where the subject is moving, such as in sports. Asdescribed above, a slow moving image with good image quality for amoving subject cannot be easily acquired in the related art.

Thus, as an effective function in a case where the normal frame rate isswitched to the slow imaging, the inventors of the present applicationpropose a method in which the switching to the slow imaging proceedsgradually (moving image is captured while increasing frame rate, andcaptured moving image is displayed at constant rate). Specifically, asdescribed in detail below, the frame rate of the moving image isgradually (continuously or stepwise, the same applies hereinafter)increased based on the movement of the subject, and the exposure time isshortened according to the change in the frame rate. In this case, thedimming degree of the dimming element (density of ND filter) isgradually changed in order to maintain the appropriate exposure inaccordance with the change in the exposure time due to the frame ratechange. Accordingly, it is possible to perform the switching to the slowimaging such that the effect is maximized at a most effective timing(for example, timing of event occurrence) according to the movement ofthe subject and thus easily acquire the slow moving image with goodimage quality for the moving subject.

<Processing of Imaging Method>

FIGS. 3 and 4 are flowcharts showing processing of the imaging methodaccording to the first embodiment.

<Setting of Imaging Condition>

The condition setting unit 342 (processor) sets an imaging condition forthe moving image (step S100: setting processing, setting step). Examplesof the imaging condition set in step S100 include the frame rate of themoving image, an exposure time per frame, the stop, initial valuesand/or target values of the degree of dimming and the like, the framerate, the exposure time, and a time to reach the target value from thestart of changing the degree of dimming (first period). The conditionsetting unit 342 may set the imaging condition in response to the user'soperation via the operation unit 370 or the monitor 390 or may set apredetermined imaging condition.

<Imaging of Normal Moving Image>

In a case where there is an imaging instruction (YES in step S110), theimaging unit 350 (processor) starts imaging and recording the movingimage (normal moving image) (step S120). At this point, the frame rateand exposure time remain at their initial values.

<Calculation of Event Occurrence Timing Based on Movement>

The detection processing unit 344 (processor) detects the movement ofthe subject based on, for example, a difference (movement vector)between frames (step S130: detection processing, detection step). In acase where the movement is detected (YES in step S130), the detectionprocessing unit 344 calculates a timing of event occurrence based on themovement.

FIGS. 5A and 5B are diagrams showing a state of calculating the eventtiming based on the movement of the subject. FIG. 5A shows a state inwhich a ball 502 and a bat 504 are shown in a field of view 500 of thecamera 100, and FIG. 5B shows a state in which the ball 502 and the bat504 are moving. The term “event” may be, for example, a state in which aplurality of subjects collide or overlap each other as viewed from thecamera 100, or a distance between the subjects is equal to or less thana threshold value (predetermined value or less). In the case of FIGS. 5Aand 5B, the detection processing unit 344 calculates the event timingwith the collision between the ball 502 and the bat 504 as the “event”.Since a batter may miss the ball (the ball 502 and the bat 504 do notcollide), the event timing may be calculated with a case where thedistance is equal to or less a threshold value, which is larger thanzero, as the “event” in addition to the case where the distance is zero(collision).

<Setting of Control Plan>

The image processing device 340 (frame rate control unit 346 andexposure control processing unit 348; processor) sets a control plan forthe frame rate and the exposure based on the event occurrence timingcalculated from the movement of the subject (step S150: frame ratecontrol, exposure control processing, frame control step, and exposurecontrol step). Specifically, the frame rate control unit 346 and theexposure control processing unit 348 set the control plan including afirst timing for starting to change the frame rate and the exposure timeand a second timing for setting the frame rate and the exposure time tothe target values (step S150: control plan setting, control plan settingstep). For example, in the case of FIGS. 5A and 5B, the frame ratecontrol unit 346 can calculate a moment at which the ball 502 and thebat 504 collide (event occurrence timing) as the second timing and atiming preceding for the first period from the second timing as thefirst timing. For example, the frame rate control unit 346 and theexposure control processing unit 348 can set the control plan such as“since the event is predicted to occur after three seconds (secondtiming), start controlling the frame rate and the exposure time afterone second (first timing) preceding for two seconds (first period) fromthe second timing and set the frame rate and the exposure time to thetarget values after three seconds (second timing)”. In a case wheremultiple times of event occurrence are predicted, any event may be setas the second timing.

The frame rate control unit 346 may calculate a timing at which a settime has elapsed from the moment at which the ball 502 and the bat 504collide (event occurrence) as the second timing. The frame rate controlunit 346 may receive a value of the “first period” designated (in stepS100) by the user in a case where the first timing is calculated or mayuse a predetermined value.

<Control of Frame Rate and Exposure Time>

The image processing device 340 (processor) starts controlling the framerate and the exposure time at the first timing (YES in step S160). FIGS.6A to 6C are graphs showing a state of controlling the frame rate andthe like. FIG. 6A shows the state of controlling the frame rate, andFIG. 6B shows the state of controlling the exposure time.

The frame rate control unit 346 (processor) continuously increases theframe rate of the moving image output from the imaging element 310 froma time point T1 (first timing) to a time point T2 (second timing) (stepS170: frame rate control, frame rate control step). In the example shownin FIG. 6A, the frame rate control unit 346 (processor) changes theframe rate of the moving image from 60 frame per second (fps; initialframe rate) to 240 fps (target value). Note that Δt (=T2−T1) correspondsto “first period”.

The exposure control processing unit 348 (processor) continuouslyshortens the exposure time per frame of the moving image from the timepoint T1 (first timing) to the time point T2 (second timing) accordingto the increase in the frame rate (step S180: exposure controlprocessing, exposure control step). At the time point T1, the exposuretime is 1/120sec (initial exposure time, Tv value (time value) is 6.91).At the time point T2, the exposure time is 1/480sec (target value, Tvvalue is 8.91). The exposure control processing unit 348 (processor)preferably sets the exposure time per frame to half or more of a frameinterval. With such a setting, it is possible to suppress or reduce thephenomenon (so-called “sense of ruffling”) in which the movement of thesubject becomes jerky and intermittent in the captured moving image. Inthe examples shown in FIGS. 6A to 6C, the exposure time is half of theframe interval, and a rate (proportion) between the frame interval andthe exposure time per frame is maintained constant.

<Control of Dimming Degree>

With the control of the frame rate and the exposure time describedabove, a moving image (first moving image, slow moving image) thatgradually slows down during reproduction can be obtained. However, in acase where only the exposure time is changed in accordance with theframe rate (control shown in FIGS. 6A and 6B), the exposure time becomesshorter and thus the obtained image becomes darker and darker. A changein the exposure can be suppressed by performing the imaging with theexposure at 1/480sec set in advance, but this will cause the “sense ofruffling” at 60 fps. In addition, the sensitivity may be changed inaccordance with the change in the exposure time. However, an S/N changesas the sensitivity changes, and thus the “sense of granularity”(presence or absence, degree, pattern, and the like of granular noise inimage) changes. Thus, as shown in FIG. 6C, the dimming controlprocessing unit 352 (processor) and the lens drive control unit 360(processor) can continuously change a density (dimming degree) of thedimming element 210 in accordance with the change in the exposure time(step S190: dimming degree control processing, dimming degree controlstep). For example, an ND value (degree of dimming) that is 16 at thetime point T1 is lowered to 4 at the time point T2. With such control ofthe dimming degree, it is possible to capture the moving image thatgradually slows down while maintaining a brightness and the sense ofgranularity constant. The frame rate, exposure time, and degree ofdimming may be changed stepwise rather than continuously.

<Recording and Displaying of First Moving Image>

The imaging unit 350 (processor) captures the moving image (first movingimage, slow moving image) under the frame rate, exposure time, anddimming degree subjected to the above control and records the image onthe recording device 380. The “first moving image” is a moving imagecaptured by increasing the frame rate after the first timing, shorteningthe exposure time, and changing the degree of dimming. In this case, theimaging unit 350 records in the header or the like of the moving imagefile that “moving image having a frame rate of 60 fps”. The displaycontrol unit 356 can reproduce and display the captured moving image onthe monitor 390 at the recorded frame rate (step S200: first imaging,first display, first imaging step, first display step). Thus, an effectof the slow moving image can be obtained. The first moving image, thesecond moving image (described below), and the normal moving image canbe recorded in the same moving image file.

The imaging unit 350 and the display control unit 356 may display themoving image acquired in this manner on the monitor 390 in real time(step S200: first imaging, first display, first imaging step, firstdisplay step). In a case where the moving image is displayed in realtime, the imaging unit 350 and the display control unit 356 display thenormal moving image on the monitor 390 during a period in which thenormal moving image is acquired (period in which the frame rate or thelike is not controlled). Further, during a period in which the firstmoving image is acquired (period in which the frame rate and the likeare controlled), the imaging unit 350 and the display control unit 356may thin out frames of the first moving image to generate the secondmoving image having an initial frame rate (the same frame rate and thelike as the normal moving image) and display the second moving image onthe monitor 390 in real time (second imaging, second display, secondimaging step, second display step). With such a display in real time,the user can capture the subject on the monitor 390. An aspect in whichthe first moving image and the second moving image are displayed at thesame time will be described in Modification Example 6 described below.

The image processing device 340 (processor; imaging unit 350 and thelike) continues to capture and record the first moving image until acontrol end timing (step S210; time point T2 which is the second timingin the examples of FIGS. 6A to 6C.

<Return to Initial Frame Rate, and Like>

After the control end timing (second timing) has elapsed (YES in stepS210), the frame rate control unit 346 and the exposure controlprocessing unit 348 return the frame rate and the exposure time tovalues before the frame rate control and the exposure control processingare started (step S220: frame rate control, frame rate control step,exposure control processing, exposure control step). Further, thedimming control processing unit 352 also returns the dimming degree toan original value. The original value may be returned immediately afterthe time point T2 has elapsed, or may be returned after a set time fromthe time point T2 has elapsed. The returning timing to the originalvalue may be set in step S100 based on the user's operation or the like.Further, in a case where there is another imaging condition to becontrolled, the image processing device 340 (processor) returns thecondition to an original value. Accordingly, the moving image to becaptured becomes the normal moving image.

The image processing device 340 (processor) repeats the above processinguntil there is an instruction to end the imaging (for example, pressingdown release button (not shown) or the like) (until YES is obtained instep S230).

<Display of Slow Moving Image>

The display control unit 356 (processor) can display the captured movingimage (normal moving image and first moving image which is slow movingimage) on the monitor 390 and/or an external display device at theinitial frame rate (the same frame rate as before starting the framerate control). Accordingly, the user can watch the slow moving imagewith good image quality (moving image that gradually slows down towardevent occurrence) for the moving subject.

As described above, in the camera 100 according to the first embodiment,the frame rate is gradually and slowly changed toward the event such asthe collision according to the movement of the subject, the exposuretime is changed accordingly, and thus the event can be effectivelyperformed. In addition, with the change of the degree of dimming of thedimming element (density of variable ND filter or the like) inaccordance with the change in the exposure time, it is possible tochange the frame rate and the exposure time without sacrificing thechange in the sense of ruffling before the frame rate changing or thesense of granularity due to the frame rate changing. As described above,with the camera 100 according to the first embodiment, it is possible toeasily acquire the slow moving image with good image quality for themoving subject.

Modification Example 1: Aspect of Controlling Stop

In the first embodiment described above, the dimming degree (variable NDfilter density) of the dimming element 210 is controlled in addition tothe control of the frame rate and the exposure time. However, instead ofor in addition to the dimming degree, the stop may be controlled. Forexample, in a case where the exposure time is controlled as shown inFIG. 6B, the stop control unit 354 (processor) can change a degree ofstop of the stop 230 (stop mechanism) as shown in FIG. 7 (graph showingstate of controlling stop) to a release side (F-number is changed from16 to 4 in the example of FIG. 7 ), in accordance with the shortening ofthe exposure time, from the time point T1 (first timing) to the timepoint T2 (second timing) (stop control processing, stop control step).In a case where the stop is controlled, the depth of field graduallybecomes shallower as the moving image slows down (frame rate increases).Thus, it is possible to further enhance the performance of the event.Whether to control the dimming degree or the stop can be decidedaccording to the user's operation (for example, set in step S100).

Modification Example 2: Aspect of Fixing Exposure Time and DimmingDegree for Fixed Period

FIGS. 8A to 8C are other graphs showing a state of controlling the framerate and the like. As described above, the exposure time is generallyset to half of “1/frame rate” (frame interval) or half or more thereofin a normal moving image. However, in Modification Example 2, in a casewhere the frame rate is changed, the value of the frame rate is fixeduntil the exposure time becomes “1/frame rate at which the exposure timecannot be secured any more”, and then the exposure time of “1/framerate” in accordance with the frame rate is set. Accompanied by the abovesetting, a density change timing and an amount of change of the dimmingelement are also changed in accordance with the change in the exposuretime.

Specifically, in a case where the frame rate is controlled as shown inFIG. 8A, the exposure time of “1/frame rate” cannot be secured after atime point TlA in a case where the exposure time remains the initialexposure time ( 1/120sec). Thus, as shown in FIGS. 8B, the exposurecontrol processing unit 348 performs the control such that the exposuretime point becomes “1/frame rate” after the time point TlA (firsttiming) (exposure time is shortened; exposure control processing,exposure control step).

Further, as shown in FIGS. 8C, the dimming control processing unit 352reduces the degree of dimming of the dimming element 210 from the timepoint TlA to the time point T2 in accordance with the control of theexposure time (dimming control processing, dimming control step). In theexample shown in the figures, the degree of dimming is reduced from 16to 8 in the ND value, and the decrease in the ND value is suppressed ascompared with the control aspect shown in FIGS. 6A to 6C. Accordingly,it is possible to suppress the influence of a variation in the dimmingelement 210 (variable ND filter).

Modification Example 3: Aspect of Setting Change Rate According toRelative Speed of Subject

In a case of imaging of a scene where a speed of the subject is known,the imaging person may perform the setting in advance. However, in acase where the speed is unknown or the speed varies, it is difficult tomake a right setting. Also, the imaging is not always possible toperform again many times. Even though the amount of change in the framerate is set in advance, in a case where the speed of the subject is toofast, a timing of changing the frame rate may have already passed whenthe subject enters the angle of view.

Thus, in Modification Example 3, with adjustment of the amount of changein the frame rate per time in accordance with the speed of the subject,it is possible to apply the effect of slowing down in accordance with atiming at which the subject just collides (or distance becomes thresholdvalue or less).

FIG. 9 is a diagram showing a state of controlling the frame rate basedon a relative speed of the subject. In the aspect shown in the figure,the frame rate control unit 346 decides the amount of change in theframe rate per time from the relative speed of the subject detected bythe detection processing unit 344. Specifically, the faster the relativespeed of the subject, the larger a rate of change in the frame rate(amount of change per time). For example, in a case where the relativespeed of the subject is larger than the control aspect shown in FIGS. 6Ato 6C (change from 60 fps to 240 fps from time point T1 to time pointT2; indicated by dotted line in FIG. 9 ), the frame rate control unit346 changes the frame rate along the solid line in FIG. 9 from the timepoint T1 to a time point T1B (before time point T2). In this case, therate of change in the frame rate is larger than the aspect in FIGS. 6Ato 6C. Depending on the relative speed of the subject, the rate ofchange in the frame rate may be smaller than the aspect in FIGS. 6A to6C. In this manner, in a case where the frame rate is controlled basedon the relative speed of the subject, it is preferable to control theexposure time, the dimming degree, the stop, and the like in accordancewith the frame rate.

Modification Example 4: Aspect of Setting Bit Rate According to FrameRate

As the frame rate increases due to the imaging of the slow moving imagedescribed above, the number of frames per second increases. In a casewhere a bit rate setting (Mbps) is left as it is in this situation, anamount of data that can be used for one frame decreases, and thus theimage quality deteriorates even though the scene is reproduced in aslowed manner. Thus, in Modification Example 4, a compression rate ischanged according to the frame rate. Specifically, the imaging unit 350captures the moving image (first moving image) by setting the bit ratehigher (lower as compression rate) as the frame rate is higher such thatthe bit rate during reproduction becomes constant. Accordingly, theproblem of image quality deterioration can be solved, and it is possibleto maintain the image quality of the slow scene and acquire the slowmoving image with good image quality.

Modification Example 5: Control in Case where Part of Subject isStationary

Although the case where the plurality of subjects are moving isdescribed in FIGS. 5A to 6C, according to the present invention, it ispossible to capture the slow moving image even in a case where some ofthe subjects are stationary. FIGS. 10A and 10B are diagrams showing theevent occurrence by a moving subject and a stationary subject. In theexamples shown in FIGS. 10A and 10B, an athlete 530 is running toward astationary goal 540, as shown in FIG. 10A. As shown in FIG. 10B, theathlete 530 approaches or passes the goal 540 as time passes (distancebetween plurality of subjects becomes equal to or less than thresholdvalue). By controlling the frame rate and the like with a timing ofpassing the goal as the “event”, the user can easily acquire the slowmoving image with good image quality for the moving subject (athlete530) in the same manner as the aspect described above. In a case wherethe slow moving image is captured by including the stationary subject(goal 540) as in the examples of FIGS. 10A and 10B, it is preferable todesignate the stationary subject in advance in response to the user'soperation via the operation unit 370, the monitor 390, or the like.

Modification Example 6: Aspect of Displaying First Moving Image andSecond Moving Image at Same Time

In a case where the slow moving image is captured, the user can capturethe subject by displaying the moving image on the monitor 390 (displaydevice) in real time as described above. However, even though the movingimage (first moving image) that changes slowly by changing the framerate or the like as described above is captured, the effect cannot beverified unless the image is reproduced after the imaging is completed.Thus, in Modification Example 6, in addition to the display of thenormal moving image and the second moving image in real time, the firstmoving image (moving image captured by reflecting slow effect) isdisplayed at the same time.

FIG. 11 is a flowchart showing processing of Modification Example 6. Inthe flowchart of the figure, the same step number is assigned to thesame processing of the flowcharts of FIGS. 3 and 4 , and detaileddescription thereof will be omitted.

The imaging unit 350 (processor) determines whether or not it is atiming to acquire the second moving image (step S202: second imaging,second imaging step). For example, in a case where the frame rate of thefirst moving image is twice the initial frame rate, one of two frames isthinned out to make the remaining frame the second moving image. In acase where the frame rate of the first moving image is three times theinitial frame rate, two of three frames are thinned out to make theremaining frame the second moving image. As the frame rate of the firstmoving image increases, the thinning rate also increases. The imagingunit 350 records the second moving image on the recording device 380,and the display control unit 356 causes the monitor 390 to display thesecond moving image (step S204: second imaging, second imaging step).

FIGS. 12A to 12C are diagrams showing a state in which the first movingimage and the second moving image are displayed at the same time. InFIG. 12A, in a display area of the monitor 390, a first moving imagedisplay area 520 is provided in a second moving image display area 510.FIG. 12A shows a state in which the control of the frame rate is started(state of first timing), and there is no difference between the firstmoving image and the second moving image. On the contrary, FIGS. 12A to12C show a state in which the frame rate is increased and the differencebetween the first moving image and the second moving image is widened.With such a display, the user can check the state of the first movingimage (slow moving image) while capturing the subject in the secondmoving image.

In the examples of FIGS. 12A to 12C, the first moving image display area520 is provided in the second moving image display area 510, butconversely, the second moving image display area 510 may be provided inthe first moving image display area 520. Further, the display controlunit 356 may display the first moving image and the second moving imageon different display areas or different display devices.

Modification Example 7: Recommendation for User

The information output unit 358 (processor) may output the informationprompting the user to set at least one of the frame rate, the exposuretime, the degree of dimming of the dimming element, the stop, thesensitivity, or the bit rate to the monitor 390, the speaker 395, or thelike, according to the set imaging condition and the situation of ascene to be captured (brightness, movement of the subject, or the like).

For example, in a situation where the imaging scene is bright and thedimming element 210 (variable ND filter) cannot be further darkened withthe exposure set by the user before the switching of the frame rate andis overexposed, the information output unit 358 can output informationprompting external attachment of the ND filter. Further, in the samesituation, the information output unit 358 can output informationprompting the stop 230 to be narrowed down or information prompting theexposure time to be shortened. With the setting of the stop and theexposure time according to such information, the user can perform theimaging with an appropriate exposure even for a bright scene.

On the contrary, in a case where the imaging scene is dark, theinformation output unit 358 can output information prompting thesensitivity to be increased, information prompting the stop to bewidened, or information prompting a fastest value (target value) of thecontrolled frame rate to be decreased (that is, degree of slowness attime of reproduction to be weakened). With the setting according to suchinformation, the user can perform the imaging with an appropriateexposure even for a dark scene.

The information output unit 358 can output such information at a timingsuch as in a case where the imaging condition is set (step S100) or in acase where the imaging instruction is issued (YES in step S110).

In a case where such information is output, the information output unit358 may also output a warning indicating “with operation or settingaccording to information, moving image different from imaging intentionmay be captured” to the monitor 390, the speaker 395, or the like. Forexample, it is possible to output the information indicating that thedepth of field changes by the change in the stop, the “sense ofruffling” changes by the change in the exposure time, or the “sense ofgranularity” changes by the change in the sensitivity. With such anoutput, it is possible to broaden a range of choices of the user, suchas “priority is given to imaging with appropriate exposure” and“priority is given to making the depth of field or the like asintended”.

Second Embodiment

Although the camera 100 which is a digital camera is described in thefirst embodiment, the configuration of the imaging device is not limitedthereto. Another imaging device of the present invention may be, forexample, a built-in type or external type camera for personal computer(PC) or a mobile terminal device having an imaging function as describedbelow.

Examples of the mobile terminal device according to the embodiment ofthe imaging device of the present invention include a mobile phone orsmartphone, a personal digital assistant (PDA), a portable game machine,and a smartwatch. Hereinafter, a smartphone will be described as anexample in detail with reference to drawings.

FIGS. 13A and 13B are external views of a smartphone 1 (imaging device,imaging device body) according to a second embodiment. FIG. 13A is afront view, and FIG. 13B is a rear view. The smartphone 1 shown in FIGS.13A and 13B has a flat housing 2 and comprises a display input unit 20in which a display panel 21 (display device) as a display unit and anoperation panel 22 (operation unit) as an input unit are integrated onone surface of the housing 2. Further, the housing 2 includes a speaker31 (speaker), a microphone 32, an operation unit 40 (operation unit),camera units 41 and 42 (imaging device), and a strobe 43. Aconfiguration of the housing 2 is not limited thereto. For example, aconfiguration in which the display unit and the input unit areindependent may be employed, or a configuration having a foldingstructure or a slide mechanism may be employed.

FIG. 14 is a diagram showing a schematic configuration of the smartphone1. As shown in FIG. 14 , the smartphone 1 comprises a wirelesscommunication unit 11, the display input unit 20, a call unit 30, theoperation unit 40, the camera units 41 and 42, the strobe 43, a storageunit an external input and output unit 60, a global positioning system(GPS) receiving unit 70, a motion sensor unit 80, and a power supplyunit 90. Further, the smartphone 1 comprises a main control unit 101(processor). A wireless communication function for performing mobilewireless communication via a base station device and a mobilecommunication network is provided as a main function of the smartphone1.

The wireless communication unit 11 performs wireless communication withthe base station device accommodated in the mobile communication networkin response to an instruction from the main control unit 101. Using suchwireless communication, various pieces of file data such as voice dataand image data, e-mail data, and the like are transmitted and received,and Web data, streaming data, and the like are received.

The display input unit 20 is a so-called touch panel in which an image(still image and/or moving image), character information, or the like isdisplayed to visually transmit information to the user and a useroperation on the displayed information is detected under control of themain control unit 101, and comprises the display panel 21 and theoperation panel 22.

In the display panel 21, a liquid crystal display (LCD), an organiclight-emitting diode (OLED), or the like is used as a display device.The operation panel 22 is a device that is placed such that an imagedisplayed on a display surface of the display panel 21 is visuallyrecognizable and detects one or a plurality of coordinates operated by afinger of the user or a conductor such as a pen. In a case where such adevice is operated by the finger of the user or the conductor such asthe pen, the operation panel 22 outputs, to the main control unit 101, adetection signal generated due to the operation. Next, the main controlunit 101 detects an operation position (coordinates) on the displaypanel 21 based on the received detection signal.

As shown in FIGS. 13A and 13B, although the display panel 21 and theoperation panel 22 of the smartphone 1 exemplified as an embodiment ofthe imaging device of the present invention integrally constitute thedisplay input unit 20, the operation panel 22 is disposed so as tocompletely cover the display panel 21. In a case where such adisposition is employed, the operation panel 22 may comprise a functionof detecting the user operation even in an area outside the displaypanel 21. In other words, the operation panel 22 may comprise adetection area (hereinafter referred to as display area) for anoverlapping portion that overlaps the display panel 21 and a detectionarea (hereinafter referred to as non-display area) for the other outeredge portion that does not overlap the display panel 21.

The call unit 30 comprises the speaker 31 and the microphone 32. Thecall unit 30 can convert a voice of the user input through themicrophone 32 into voice data that can be processed by the main controlunit 101 and output the converted voice data to the main control unit101, and can decode the voice data received by the wirelesscommunication unit 11 or the external input and output unit 60 andoutput the decoded voice data from the speaker 31. Further, as shown inFIGS. 13A and 13B, it is possible to mount the speaker 31 on the samesurface as a surface on which the display input unit 20 is provided, andto mount the microphone 32 on a side surface of the housing 2, forexample.

The operation unit 40 is a hardware key using a key switch or the likeand receives the instruction from the user. For example, as shown inFIGS. 13A and 13B, the operation unit 40 is a push-button type switchthat is mounted on the side surface of the housing 2 of the smartphone1, is turned on in a case of being pressed with a finger or the like,and is turned off by restoring force of a spring or the like in a casewhere the finger is released.

The storage unit 50 (recording device) stores a control program orcontrol data of the main control unit 101, application software, addressdata in which a name, a telephone number, and the like of acommunication partner are associated, data of transmitted and receivede-mails, Web data downloaded by Web browsing, or downloaded contentdata, and temporarily stores streaming data or the like. The storageunit 50 is configured of an internal storage unit 51 built into thesmartphone and an external storage unit 52 having an attachable anddetachable slot for external memory. Each of the internal storage unit51 and the external storage unit 52 constituting the storage unit 50 isformed by using a known storage medium.

The external input and output unit 60 serves as an interface with allexternal devices connected to the smartphone 1. The smartphone 1 isdirectly or indirectly connected to another external device via theexternal input and output unit 60 by communication or the like. Examplesof a unit for communication or the like include a universal serial bus(USB), IEEE1394, and a network (for example, the Internet and a wirelessLAN). In addition, examples of the unit for communication or the likeinclude Bluetooth (registered trademark), radio frequency identification(RFID), and infrared communication (infrared data association: IrDA)(registered trademark). Further, examples of the unit for communicationor the like include ultra wideband (UWB) (registered trademark) andZigBee (registered trademark).

Examples of the external device connected to the smartphone 1 include awired/wireless headset, a wired/wireless external charger, and awired/wireless data port. Further, examples of the external deviceinclude a memory card and a subscriber identity module (SIM) card/useridentity module (UIM) card connected via a card socket. Further, theexternal device connected thereto includes external audio and videodevices connected via audio and video input/output (I/O) terminals,external audio and video devices wirelessly connected, smartphoneswired/wirelessly connected, PDAs wired/wirelessly connected, personalcomputers wired/wirelessly connected, and earphones. The external inputand output unit 60 can transmit the data transmitted from such anexternal device to each component inside the smartphone 1 or cantransmit the data inside the smartphone 1 to the external device.

The motion sensor unit 80 comprises, for example, a triaxialacceleration sensor or an inclination sensor and detects a physicalmovement of the smartphone 1 in response to the instruction from themain control unit 101. With the detection of the physical movement ofthe smartphone 1, a moving direction, acceleration, or posture of thesmartphone 1 is detected. Such a detection result is output to the maincontrol unit 101. The power supply unit 90 supplies electric poweraccumulated in a battery (not shown) to each unit of the smartphone 1 inresponse to the instruction from the main control unit 101.

The main control unit 101 comprises a microprocessor and operatesaccording to the control program or the control data stored in thestorage unit 50 to integrally control each unit of the smartphone 1including the camera unit 41. The main control unit 101 has a mobilecommunication control function for controlling each unit of acommunication system and an application processing function forperforming voice communication or data communication through thewireless communication unit 11.

The main control unit 101 also has an image processing function such asdisplaying a moving image on the display input unit 20 based on theimage data (data of still image or moving image) such as received dataor downloaded streaming data. The image processing function means afunction of the main control unit 101 decoding the image data,performing the image processing on such a decoding result, anddisplaying an image on the display input unit 20.

The camera units 41 and 42 are digital cameras (imaging devices) thatperform electronic imaging using the imaging element such as CMOS orCCD. Further, the camera units 41 and 42 can convert the image data(moving image, still image) obtained by imaging into compressed imagedata such as MPEG or JPEG, and record the compressed image data in thestorage unit 50 or output the compressed image data through the externalinput and output unit 60 and the wireless communication unit 11, underthe control of the main control unit 101. Further, the camera units 41and 42 can also capture the slow moving images (first moving image,second moving image) under the control of the main control unit 101. Inthe smartphone 1 shown in FIGS. 13A, 13B, and 14 , one of the cameraunits 41 and 42 can be used for imaging, or the camera units 41 and 42can be used at the same time for imaging. In a case where the cameraunit 42 is used, the strobe 43 can be used. The camera units 41 and 42are provided with a dimming element in which the dimming degree isvariable, as in the camera 100 according to the first embodiment (notshown in FIGS. 13A, 13B, and 14 ). The dimming element (ND filter or thelike) may be an external type. A stop mechanism in which the degree ofstop is variable may be further provided.

The camera units 41 and 42 can be used for various functions of thesmartphone 1. For example, the smartphone 1 can display images acquiredby the camera units 41 and 42 on the display panel 21. Further, thesmartphone 1 can use the images of the camera units 41 and 42 as one ofthe operation inputs of the operation panel 22. Further, in a case wherethe GPS receiving unit 70 detects a position based on positioninginformation from GPS satellites ST1, ST2, . . . , and STn, thesmartphone 1 detects the position by referring to the images from thecamera units 41 and 42. Furthermore, the smartphone 1 can determine anoptical axis direction of the camera unit 41 of the smartphone 1 or acurrent use environment without using the triaxial acceleration sensoror in combination with the triaxial acceleration sensor with referenceto the images from the camera units 41 and 42. Of course, the smartphone1 can use the images from the camera units 41 and 42 in the applicationsoftware. In addition, the smartphone 1 can add the position informationacquired by the GPS receiving unit 70, voice information acquired by themicrophone 32 (may be text information subjected to voice-textconversion by the main control unit or the like), posture informationacquired by the motion sensor unit 80, and the like to the image data ofthe still image or the moving image and record the image data thereofwith the added information in the storage unit 50. Further, thesmartphone 1 can also output the image data of the still image or themoving image through the external input and output unit 60 and thewireless communication unit 11.

As in the camera 100 according to the first embodiment, the smartphone 1having the above configuration also can execute the processing(detection of subject movement, imaging of first and second movingimages, control of frame rate and exposure time, control of dimmingdegree and degree of stop, and the like) of the imaging method accordingto the present invention. Specifically, the processing (includingprocessing of flowcharts of FIGS. 3, 4, 11 ) executed by the imageprocessing device 340 (each part shown in FIG. 2 ) in the firstembodiment is mainly executed by the camera units 41 and 42 and the maincontrol unit 101 in the smartphone 1. In addition, the functions of theoperation unit 370, the recording device 380, the monitor 390, and thespeaker 395 in the first embodiment can be respectively realized by theoperation unit 40, the storage unit 50 and the operation panel 22, thedisplay panel 21 and the operation panel 22, and the speaker 31 in thesmartphone 1.

Accordingly, it is possible to easily acquire the same effect (easilyacquire slow moving image with good image quality) as that of the camera100 according to the first embodiment, also in the smartphone 1according to the second embodiment.

Although the embodiments of the present invention have been describedabove, the present invention is not limited to the above-describedaspects, and various modifications can be made without departing fromthe spirit of the present invention.

EXPLANATION OF REFERENCES

-   -   1: smartphone    -   2: housing    -   11: wireless communication unit    -   20: display input unit    -   21: display panel    -   22: operation panel    -   30: call unit    -   31: speaker    -   32: microphone    -   40: operation unit    -   41: camera unit    -   42: camera unit    -   43: strobe    -   50: storage unit    -   51: internal storage unit    -   52: external storage unit    -   60: external input and output unit    -   70: GPS receiving unit    -   80: motion sensor unit    -   90: power supply unit    -   100: camera    -   101: main control unit    -   200: interchangeable lens    -   210: dimming element    -   220: zoom lens    -   230: stop    -   240: focus lens    -   250: lens drive unit    -   300: imaging device body    -   310: imaging element    -   320: AFE    -   330: A/D converter    -   340: image processing device    -   342: condition setting unit    -   344: detection processing unit    -   346: frame rate control unit    -   348: exposure control processing unit    -   350: imaging unit    -   352: dimming control processing unit    -   354: stop control unit    -   356: display control unit    -   358: information output unit    -   360: lens drive control unit    -   362: ROM    -   364: RAM    -   370: operation unit    -   380: recording device    -   390: monitor    -   395: speaker    -   500: field of view    -   502: ball    -   504: bat    -   510: second moving image display area    -   520: first moving image display area    -   530: athlete    -   540: goal    -   L: optical axis    -   S100 to S230: each step of imaging method    -   ST1: GPS satellite    -   ST2: GPS satellite

What is claimed is:
 1. An imaging device comprising: an optical system;an imaging element; and a processor, wherein the processor performsdetection processing of detecting a movement of a subject based on animage signal output from the imaging element, frame rate control ofincreasing a frame rate of a moving image output from the imagingelement based on the detected movement, exposure control processing ofmaintaining a rate of an exposure time per frame of the moving imageconstant according to the increase in the frame rate, dimming controlprocessing of changing a degree of dimming of the optical systemaccording to the exposure control processing, and information outputprocessing of, according to an imaging situation of the subject,prompting a user to change an imaging condition set to the imagingdevice.
 2. The imaging device according to claim 1, wherein theprocessor calculates a first timing for starting to change the framerate and the exposure time and a second timing for setting the framerate and the exposure time as target values based on the movement,increases the frame rate from the first timing to the second timing, andshortens the exposure time from the first timing to the second timing.3. The imaging device according to claim 2, wherein the processorcalculates a timing at which a distance between a plurality of subjectsis equal to or less than a threshold value as the second timing based onthe detected movement in the frame rate control.
 4. The imaging deviceaccording to claim 3, wherein the processor sets a rate of change in theframe rate and the exposure time according to a relative speed betweenthe plurality of subjects.
 5. The imaging device according to claim 2,wherein the processor sets the target values for the frame rate and theexposure time.
 6. The imaging device according to claim 2, wherein theprocessor returns values of the frame rate, the exposure time, and thedegree of dimming to values set before starting the frame rate control,the exposure control processing, and the dimming control processingafter the second timing has elapsed.
 7. The imaging device according toclaim 1, wherein the processor captures a first moving image, which is amoving image of the increased frame rate after the first timing, bysetting a bit rate higher as the frame rate is higher.
 8. The imagingdevice according to claim 7, wherein the processor sets an exposure timeper frame to half or more of a frame interval for the first movingimage.
 9. The imaging device according to claim 7, wherein the processordisplays the first moving image on a display device at the same initialframe rate as before starting the frame rate control.
 10. The imagingdevice according to claim 9, wherein the processor thins out a frame ofthe first moving image to generate a second moving image having theinitial frame rate and displays the first moving image and the secondmoving image on the display device at the initial frame rate.
 11. Theimaging device according to claim 1, wherein the imaging conditionincludes at least one of a frame rate, an exposure time, a degree ofdimming of a dimming element, a stop, a sensitivity, or a bit rate. 12.The imaging device according to claim 1, wherein the optical system hasa stop mechanism in which a degree of stop is variable, and theprocessor changes the degree of stop of the stop mechanism to a releaseside in accordance with shortening of the exposure time.
 13. An imagingmethod by an imaging device including an optical system, an imagingelement, and a processor, processing performed by the processorcomprising: a detection step of detecting a movement of a subject basedon an image signal output from the imaging element; a frame rate controlstep of continuously increasing a frame rate of a moving image outputfrom the imaging element based on the detected movement; an exposurecontrol step of maintaining a rate of an exposure time per frame of themoving image constant according to the increase in the frame rate; adimming control step of changing a degree of dimming of the opticalsystem according to the control of the exposure time; and an informationoutput step of, according to an imaging situation of the subject,prompting a user to change an imaging condition set to the imagingdevice.
 14. A non-transitory, computer-readable tangible recordingmedium which records thereon, a program for causing, when read by acomputer, the computer to perform the imaging method according to claim13.